Stationary Exponential Families

Exponential families

Optimally approximating exponential families

This article studies exponential families E on finite sets such that the information divergence D(P E) of an arbitrary probability distribution from E is bounded by some constant D > 0. A particular class of low-dimensional exponential families that have low values of D can be obtained from partitions of the state space. The main results concern optimality properties of these partition exponent...

Asymptotics of Exponential Families

if we use the convention that additive terms that do not contain the parameter may be dropped from log likelihoods (because they do not affect any likelihood-based statistical inferences). Thus we have another definition of exponential families: a statistical model is an exponential family of distributions if there exists a statistic Y and a parameter θ such that the log likelihood has the form...

Satisfiability with Exponential Families

Fix a set S ⊆ {0, 1}∗ of exponential size, e.g. |S ∩ {0, 1}n| ∈ Ω(αn), α > 1. The S-SAT problem asks whether a propositional formula F over variables v1, . . . , vn has a satisfying assignment (v1, . . . , vn) ∈ {0, 1}n ∩S. Our interest is in determining the complexity of S-SAT. We prove that S-SAT is NP-complete for all context-free sets S. Furthermore, we show that if S-SAT is in P for some e...

Deep Exponential Families

We describe deep exponential families (DEFs), a class of latent variable models that are inspired by the hidden structures used in deep neural networks. DEFs capture a hierarchy of dependencies between latent variables, and are easily generalized to many settings through exponential families. We perform inference using recent “black box” variational inference techniques. We then evaluate variou...